Method and system for providing personalized on-location information exchange

ABSTRACT

A method includes: deploying a frontend system (FIES); registering a current deployment location of the FIES; detecting a close-proximity interaction between a user and the FIES that specifies a first category; generating a product recommendation that includes a first product of the first category; detecting a respective prior purchase record of the user for a second product in a second category associated with the first category; if the prior purchase record indicates a purchase or delivery location outside of a geographic region of the registered current deployment location, automatically augmenting the product recommendation to include a third product from the second category selected based on characteristics of the second product; and otherwise, automatically refining the first product recommendation to further define characteristics of the first product based on the characteristics of the second product; and providing the product recommendation to the first user after the automatic augmenting or refining.

CROSS REFERENCE TO RELATED APPLICATION

This application claims priority under § 119(e) and the benefit of U.S.Provisional Application. No. 62/572,238, filed Oct. 13, 2017, thedisclosure of which is incorporated herein in its entirety.

FIELD OF THE TECHNOLOGY

The present disclosure relates to integrating computer-based informationprocessing with in-person close-proximity physical interactions tosupport customized, on-location man-and-machine information exchange,and in particular, to a method and system for providing personalized,on-location information exchange with a human user regarding appliancerecommendations.

BACKGROUND OF THE TECHNOLOGY

In a day and age where e-commerce become increasing popular, manyproduct venders devote a large amount of resources on developing andusing on-line sales platforms that present images of products in alisting, and facilitate sales by providing product descriptions, onlinereviews, and information videos on individual product pages. Many of theon-line sales platforms also provide product search functions thatidentify subsets of all available products based on search keywordsentered by a user. Although online sales platforms also provide anavenue for sales of home appliances, pure online sales platforms cannotmeet users' desire to try out home appliances, to learn about their manyfeatures, to touch and manipulate the home appliances in person, or tosee the home appliance operating in a physical environment that mimicsthe intended operating environment of the users' homes. Pure online saleplatforms also cannot provide a user with any real-time, personalizedattention and assistance at a location where the users have thein-person close-proximity experiences with the appliances.

Brick and mortar stores are becoming increasingly rare and costly tooperate. Good and efficient sales staff are not only difficult to findbut also expensive to maintain. In addition, one sales person maydevelop a good rapport and win a sale with one customer, but may faceavoidance and rejection with respect to a different customer. In theshort amount of time that a customer typically spends inside a brick andmortar store, a real human sales person simply does not have thepersonality, energy, capacity, or knowledge to provide trulypersonalized service and assistance to each customer that walks throughthe door.

Recently, there has been a great deal of interest in developing in-storesales robots. However, many of the state-of-the-art in-store salesrobots that have been contemplated are crude combinations of theexisting e-commerce backend system with added natural languageprocessing capabilities. The sales robots simply mimic a crude naturallanguage exchange with a customer based on generic language models andkeyword identifications, and are not much more effective than theexisting online sales platforms. Generic statistical modeling of masscustomer data on the backend system also does not take into account ofan individual customer's unique in-store experience with the products,and does not take into account the uniqueness of appliance shopping incomparison with shopping of other types of products or merchandise. Theadvancement of in-store sales robots are also severely limited by theavailability of relevant data and the lack of efficient and effectiveways of selecting relevant parameters and prioritizing data processingand storage tasks.

Therefore, an improved method and system of providing personalized,on-location information exchange regarding home appliances that is fast,efficient, and effective is needed.

SUMMARY

As discussed in the background, many state-of-the art sales robots orin-store kiosks rely on statistical modeling of mass sales data andattempt to place individual users into crude categories as opposed toproviding truly personalized information exchange with the individualusers. In addition, the computer-based information processing and anindividual user's in-person on-location experience are either completelydecoupled from each other or simply combined based on a crude additivestrategy without creating any synergy between the two. The integrationbetween the user's in-store experience and the backend computer-basedinformation processing capabilities needs to be well thought out, andtailored to each individual user, and to the uniqueness of the user'spast and present circumstances. By dynamically changing how thedifferent artificial intelligence strategies and models interact witheach other, rearranging their relative roles in the overall modelretraining, user characteristic prediction, product recommendationgeneration, and direct information exchange processes, and readjustingtheir relative importance in the overall model retraining, usercharacteristic prediction, product recommendation generation, and directinformation exchange processes, the information exchange system (abackend server system and a frontend sales robot or kiosk) can providebetter user experience and better sales results when it is engaged by auser at a physical deployment location (e.g., a brick and mortar storeor exhibition hall) where real products are on display and available foruser manipulation and inspection.

The system and method disclosed herein take on the technical challengesto address the deficiencies of conventional ecommerce platforms and thedeficiencies of brick and mortar stores. In some embodiments, a frontendinformation exchange system (e.g., a sales robot or kiosk) is deployedat a deployment location (e.g., a brick and mortar store or exhibitionhall) and is used to acquire and optionally process data inputs throughdirect, in-person close-proximity interactions with human users; and abackend information exchange system is used to generate artificialintelligence models, rules, and decision logic, and to derive relevantparameters and effectively integrate those parameters with otherrelevant data that is made available through computer-based informationprocessing capabilities of the backend information exchange system.Specifically, the backend information exchange system is designed toidentify particular data that, when combined with the relevantinformation and parameters obtained through the in-personclose-proximity interactions with the human users, enable previouslyunavailable decision paths and enhance predictability of userpreferences and intent for the particular user in question. In addition,the parameters obtained through in-person close-proximity interactionswith the users also enable the backend information exchange system tofurther dynamically prioritize data storage and data processing needsfor the present information exchange tasks, and thereby making theinformation exchange system more efficient, and lessresource-constrained. In the end, the interactions and the results ofthe interactions are processed to further refine the persona of the userand the prediction models that are used for recommending productsinteracting with the user.

As disclosed herein, in some embodiments, a method of providinginformation exchange (e.g., providing machine-generated productrecommendations, such as recommendations for home appliances) includes:at an electronic device (e.g., a server hosting a product recommendationengine) having one or more processors, and memory: deploying a frontendinformation exchange system, wherein the frontend information exchangesystem provides an input user interface configured to detect one or morein-person close-proximity interactions between a user and the frontendinformation exchange system (e.g., interactions by voice, touch, sight,gesture, etc. that are enabled by the close physical proximity betweenthe user and the frontend information exchange system); registering acurrent deployment location of the frontend information exchange system;detecting a first in-person close-proximity interaction between a firstuser and the frontend information exchange system at the currentdeployment location of the frontend information exchange system, whereinthe first in-person close-proximity interaction specifying a firstproduct category for which the first user is seeking productrecommendations; in response to the first in-person close-proximityinteraction between the first user and the frontend information exchangesystem: automatically generating a first product recommendation, whereinthe first product recommendation includes a first product of the firstproduct category that is selected based at least in part on theregistered current deployment location of the frontend informationexchange system; detecting a respective prior purchase record of thefirst user for a second product in a second product category that isassociated with the first product category; in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location outside of apredefined geographic region of the registered current deploymentlocation of the frontend information exchange system, automaticallyaugmenting the first product recommendation to include a third productfrom the second product category that is selected based oncharacteristics of the second product; and in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of thefrontend information exchange system, automatically refining the firstproduct recommendation to further define one or more characteristics ofthe first product based on the characteristics of the second product;and providing, through the frontend information exchange system, thefirst product recommendation to the first user after the automaticaugmenting or refining of the first product recommendation.

As disclosed herein, in some embodiments, a method of providinginformation exchange (e.g., providing machine-generated productrecommendations, such as recommendations for home appliances) includes:at an electronic device (e.g., a server hosting a product recommendationengine) having one or more processors, and memory: A method of providingmachine-generated product recommendations, comprising:

at an electronic device (e.g., a server hosting a product recommendationengine) having one or more processors, and memory: deploying a frontendinformation exchange system, wherein the frontend information exchangesystem provides an input user interface configured to detect one or morein-person close-proximity interactions between a user and the frontendinformation exchange system (e.g., interactions by voice, touch, sight,gesture, etc. that are enabled by the close physical proximity betweenthe user and the frontend information exchange system); activatingrespective input streams from one or more on-site cameras located at acurrent deployment location of the frontend information exchange system,wherein each of the one or more on-site cameras is located in proximityto a respective sample product on display at the current deploymentlocation of the frontend information exchange system; in accordance withthe respective input stream of a first camera of the one or more on-sitecameras, registering a first inspection event of a first user inassociation with a first sample product on display at the currentdeployment location of the frontend information exchange system;detecting a first in-person close-proximity interaction between thefirst user and the frontend information exchange system at the currentdeployment location of the frontend information exchange system; inresponse to the first in-person close-proximity interaction between thefirst user and the frontend information exchange system, automaticallygenerating a first product recommendation, including: in accordance witha determination that the first inspection event of the first user thathas been registered in association with the first sample product meetsenhanced inspection criteria, wherein the enhanced inspection criteriainclude a criterion that is met when at least a second inspection eventof the first user exists in a plurality of previously stored inspectionevents associated with the respective first sample product,automatically adding a product-specific description of the first sampleproduct in the first product recommendation; and in accordance with adetermination that the first inspection event of the first user that hasbeen registered in association with the first sample product does notmeet the enhanced inspection criteria, forgoing including theproduct-specific description of the first sample product in the firstproduct recommendation; and providing, through the frontend informationexchange system, the first product recommendation to the first user.

In accordance with some embodiments, an electronic device includes oneor more processors, and memory storing one or more programs; the one ormore programs are configured to be executed by the one or moreprocessors and the one or more programs include instructions forperforming or causing performance of the operations of any of themethods described herein. In accordance with some embodiments, acomputer readable storage medium has stored therein instructions, which,when executed by an electronic device, cause the device to perform orcause performance of the operations of any of the methods describedherein. In accordance with some embodiments, an electronic deviceincludes means for performing or causing performance of the operationsof any of the methods described herein. In accordance with someembodiments, an information processing apparatus, for use in anelectronic device includes means for performing or causing performanceof the operations of any of the methods described herein.

Various additional advantages of the present application are apparent inlight of the descriptions below.

BRIEF DESCRIPTION OF THE DRAWINGS

The aforementioned features and advantages of the disclosed technologyas well as additional features and advantages thereof will be moreclearly understood hereinafter as a result of a detailed description ofpreferred embodiments when taken in conjunction with the drawings.

To describe the technical solutions in the embodiments of the presentdisclosed technology or in the prior art more clearly, the followingbriefly introduces the accompanying drawings required for describing theembodiments or the prior art. Apparently, the accompanying drawings inthe following description show merely some embodiments of the presentdisclosed technology, and persons of ordinary skill in the art may stillderive other drawings from these accompanying drawings without creativeefforts.

FIG. 1A is a block diagram illustrating an operating environment of aninformation exchange system, in accordance with some embodiments.

FIG. 1B is a block diagram illustrating exemplary components of abackend information exchange system in accordance with some embodiments.

FIG. 1C is a block diagram illustrating exemplary components of afrontend information exchange system and its deployment environment inaccordance with some embodiments.

FIG. 2 is example user interface of a frontend information exchangesystem in accordance with some embodiments.

FIG. 3 is a flowchart diagram of a method of providing informationexchange in accordance with some embodiments.

FIG. 4 is a block diagram of a server or backend information exchangesystem in accordance with some embodiments.

FIG. 5 is a block diagram of a client device or frontend informationexchange system in accordance with some embodiments.

Like reference numerals refer to corresponding parts throughout theseveral views of the drawings.

DESCRIPTION OF EMBODIMENTS

Reference will now be made in detail to embodiments, examples of whichare illustrated in the accompanying drawings. In the following detaileddescription, numerous specific details are set forth in order to providea thorough understanding of the subject matter presented herein. But itwill be apparent to one skilled in the art that the subject matter maybe practiced without these specific details. In other instances,well-known methods, procedures, components, and circuits have not beendescribed in detail so as not to unnecessarily obscure aspects of theembodiments.

The following clearly and completely describes the technical solutionsin the embodiments of the present application with reference to theaccompanying drawings in the embodiments of the present application. Thedescribed embodiments are merely a part rather than all of theembodiments of the present application. All other embodiments obtainedby persons of ordinary skill in the art based on the embodiments of thepresent application without creative efforts shall fall within theprotection scope of the present application.

As shown in FIG. 1A, an information exchange system 100 (e.g., includinga backend information exchange system 108 and one or more frontendinformation exchange system 104) is implemented in accordance with aserver-client interaction model in accordance with some embodiments. Inaccordance with some embodiments, the server-client interaction modelincludes client-side modules 102-1, 102-2, etc. (also referred to as the“frontend modules 102”) executed on a frontend information exchangesystem 104-1, 104-2, etc. that are deployed at various deploymentlocations (e.g., brick and mortar stores, roadshow booths, productdemonstration sites, exhibition halls, etc.). In some embodiments, theserver-client interaction model further includes a server-side module106 (also referred to as the “backend module 106”) executed on a backendserver system (e.g., backend information exchange system 108). Thefrontend modules 102 communicate with the backend module 106 through oneor more networks 110. The frontend modules 102 provide user-sidefunctionalities for the information exchange system 100 and communicatewith the backend module 106. The backend module 106 provides server-sidefunctionalities for the information exchange system 100 for any numberof frontend modules 102 each residing on a frontend information exchangesystem 104 (e.g., an on-site kiosk, sales robot, etc.).

In some embodiments, the backend module 106 includes one or moreprocessors 112, various proprietary databases (e.g., databases 114 formarket analysis data, and databases 116 for customer transactions,databases for current customer interaction sessions, and databases foruser profiles, etc.), I/O interface 118 to one or more frontend modules,and an I/O interface 120 to one or more external services or otherindividual online interactions (e.g., user interacting with theinformation exchange system 100 through pure online sales channel (e.g.,ecommerce or social networking apps 105) on their individual userdevices 103 (e.g., smart phones, tablet devices, and personalcomputers)).

In some embodiments, the I/O interface 118 to frontend modulesfacilitates the client-facing input and output processing for thefrontend modules 106. The I/O interface 118 further includes inputprocessing for various peripheral equipment such as on-site cameras, GPStrackers, and/or on-site microphones distributed throughout (e.g., overeach product on display, or over product display region, along eachaisle, etc.) each of various deployment locations of the frontendinformation exchange systems 104. In some embodiments, the backendinformation exchange system 108 directly communicates with and controlsthe on-site peripheral equipment. In some embodiments, the backendinformation exchange system 108 communicates with and controls theon-site peripheral equipment via the frontend information exchangesystems located at the various deployment sites where the on-siteperipheral equipment are located respectively.

In some embodiments, the databases 114 store various market analysisdata (e.g., factors that are influential to sales and profits, customerperception, strategic sales and marketing planning, etc.) that isstatistical or aggregated in nature and represents summaries and trendsof past marketing, sales, and research results. The databases 116 storeindividual transaction records organized in various searchable formats(e.g., by customer name, age, income level, color preference, previouslypurchased product, product category, product combination/bundle,previous inquired product, past delivery location, interaction channel,sales representative, location of interaction, purchase time, deliverytime, customer comments, special requests, etc.). Other databasesinclude a user profile database that includes searchable characteristicsfor each user (e.g., identity data, demographic data, socialrelationships, social network account names, social network publicationor comments, interaction records with sales representatives, customerservice representatives, or delivery personnel, preferences, dislikes,sentiment, beliefs, superstitions, personality, temperament, interactionstyle, etc.).

In some embodiments, the databases 116 stores user profile data. Theuser profile data includes integrated personas that are generated basedon self-descriptions directly obtained from the users (e.g., via salesrecords, or evaluations, surveys, etc.), descriptions on social networksor public records (e.g., publications, comments, social networkinterests, etc.), descriptions provided through direct humaninteractions and human observations (e.g., descriptions recorded bysales representatives, delivery personnel, customer servicerepresentatives, such as age, weight, living environment, personality,temperament, interests, human interaction style and preferences,personal styles and tastes, etc.), images and descriptions of the images(e.g., images of the user's visits to the stores, images posted onsocial media, etc., including which products the user looked at,interacted with, whether the user was with family, friends, or alone,how the user looked, the mood of the user, etc.), the user's browsingroute and browsing styles such as whether the user directly looked atthe products of interest or prefers to roam and browse in a wide varietyof product categories, whether the user engaged any sales personnelduring his/her visit, etc.

The other databases also include a session database that temporarilystores data that is currently collected through on-site interactionswith individual users (e.g., images of each user that are captured whilethe user is browsing through the different products on display along thevarious aisles, images of the user when the user is interacting directlywith the frontend information exchange system 104 that has been deployedat the store, speech interactions or textual exchanges between the userand the frontend information exchange system 104, the browsing route andpattern of the user in the store, browsing and viewing of electronicinformation (e.g., interaction with the product description or demos atthe store for each product) at the store, virtual reality and augmentedreality interactions the user had with the product on display, etc.).

In some embodiments, the I/O interface 120 to one or more externalservices facilitates communications with one or more external services122 (e.g., web servers or cloud-based service providers such as filesharing, data storage services, data collection and dispensary services,data forecast services, etc.). For example, the external services 122optionally include a weather forecast service and provide current,future, and historic weather data for various locations and dates. Theexternal services 122 optionally include intelligence data on sales andpromotion strategies for competitors or related products. The externalservices 122 optionally include fortune forecasting and historic fortuneforecast records, or principles of various forecast or belief systems.Each of these external services optionally provide data that is used totrain a prediction model, provide basis for a statistical analysis,provide basis for establishing a personal profile for a particular user,and/or used in a current prediction or analysis using existing models,rules, and statistical analysis results.

Examples of the frontend information exchange system 104 include, butare not limited to, an on-site computer kiosk, a mobile sales robot, ahumanoid robot, a centrally-controlled display terminal, a handheldcomputer, a wearable computing device, a personal digital assistant(PDA), a tablet computer, a laptop computer, a desktop computer, acellular telephone, a smart phone, an enhanced general packet radioservice (EGPRS) mobile phone, a media player, a navigation device, agame console, a television, a remote control, a point of sale (POS)terminal, vehicle-mounted computer, an ebook reader, or a combination ofany two or more of these data processing devices or other dataprocessing devices.

Examples of one or more networks 110 include local area networks (LAN)and wide area networks (WAN) such as the Internet. One or more networks110 are, optionally, implemented using any known network protocol,including various wired or wireless protocols, such as Ethernet,Universal Serial Bus (USB), FIREWIRE, Long Term Evolution (LTE), GlobalSystem for Mobile Communications (GSM), Enhanced Data GSM Environment(EDGE), code division multiple access (CDMA), time division multipleaccess (TDMA), Bluetooth, Wi-Fi, voice over Internet Protocol (VoIP),Wi-MAX, or any other suitable communication protocol.

The backend information exchange system 108 is implemented on one ormore standalone data processing apparatuses or a distributed network ofcomputers. In some embodiments, the backend information exchange system108 also employs various virtual devices and/or services of third-partyservice providers (e.g., third-party cloud service providers) to providethe underlying computing resources and/or infrastructure resources ofthe backend information exchange system 108. In some embodiments, thebackend information exchange system 108 includes, but is not limited to,a handheld computer, a tablet computer, a laptop computer, a desktopcomputer, a server computer, or a combination of any two or more ofthese data processing devices or other data processing devices.

The backend information exchange system 108 also implements variousmodules for supporting the frontend interactions and productrecommendations to the user located at different frontend deploymentlocations. In some embodiments, the backend information exchange systemincludes audio/video processing services, natural language processingservices, model building services, statistical analysis services, datamining services, data collection services, and product recommendationservices, etc., based on various statistical techniques, rule-basedtechniques, and artificial intelligence-based techniques.

The information exchange system 100 shown in FIG. 1A includes both aclient-side portion (e.g., the frontend module 102) and a server-sideportion (e.g., the backend module 106). In some embodiments, dataprocessing is implemented as a standalone application installed on afrontend device 104 that is deployed at a deployment location thatphysically displays a plurality of actual products (e.g., homeappliances, furniture, heavy equipment, vehicles, etc.), where the useris physically present at the location and directly interacts with thefrontend device(s) and the products. In addition, the division offunctionalities between the client and server portions of informationexchange system 100 can vary in different embodiments. For example, insome embodiments, the frontend module 102 is a thin-client that providesonly user-facing input and output processing functions, and delegatesall other data processing functionalities to a backend server (e.g., thebackend information exchange system 108). Although many aspects of thepresent technology are described from the perspective of the backendsystem, the corresponding actions performed by the frontend system wouldbe apparent to ones skilled in the art without any creative efforts.Similarly, although many aspects of the present technology are describedfrom the perspective of the frontend system, the corresponding actionsperformed by the backend system would be apparent to ones skilled in theart without any creative efforts. Furthermore, some aspects of thepresent technology may be performed by the server, the client device, orthe server and the client cooperatively. In some embodiments, some ofthe databases (e.g., user profile data for users with known homelocations) are distributed at various locations that are local to someof the frontend systems, which enable faster data access and local dataprocessing time.

FIG. 1B is a block diagram illustrating exemplary components of thebackend information exchange system 108 in accordance with someembodiments. In FIG. 1B, the backend information exchange system 108includes a plurality of modules, including e-commerce data processingmodule 152, purchase/delivery/service history processing module 153,historic human interaction log processing module 154, current in-personinteraction processing module 156, data processing integration module158, augmented reality/virtual reality processing module 160, productrecommendation module 162, natural language processing module 164, imageprocessing module 166, external data acquisition and processing module168, statistical modeling module 170, deep learning module 172, I/Oprocessing module 174, and other modules 176 (e.g., satisfactionscoring, demographic determination, etc.) and sub-modules. The data thatis accessible to the backend information exchange system 108 includeproduct data 178, market data, 180, user data 182, session data 184,external event data 186, historic sales record data 188, humaninteraction log 190, and other data 192 (e.g., belief data, policy data,etc.). These modules utilizes the various real-time data obtainedthrough various internal and external services, real-time data receivedfrom the frontend information exchange systems, and existing data storedin the various databases, to guide the in-person interactions with theusers at various deployment locations of the frontend informationexchange systems 104 and generate product recommendations to the users.

FIG. 1C is a block diagram illustrating exemplary components of thefrontend information exchange system 104 in accordance with someembodiments. The frontend system 104 is deployed at a deploymentlocation (e.g., a brick and mortar store or display venue) where manyproducts (e.g., products such as home appliances 138) are physically ondisplay. A user (e.g., user 139) can walk around the deployment locationand physically inspect the products on display. Peripheral equipmentsuch as GPS trackers, microphones, and cameras (e.g., cameras 136) aredistributed throughout the deployment location, e.g., in proximity todifferent products, different product display locations, and/ordifferent product categories, where the output streams from theperipheral equipment captures the location, voice, and images of theusers at the various parts of the deployment location. The frontendinformation exchange system 104 may be a robot with a humanoid form, ora computer-kiosk with a display and a speaker, or somewhere in between.A user (e.g., user 139) can walk close up to the frontend informationexchange system and directly engage in in-person, close proximityinteractions with the frontend information exchange system, e.g.,through speed, gesture, facial expressions, touch, movement, keyboardentry, or other physical manipulations, that are enabled due to theclose physical proximity between the user and the frontend informationexchange system.

In FIG. 1C, the frontend information exchange system 104 includes one ormore processors 104, a plurality of interfaces and modules, includingI/O interface 124 to users, I/O interface 126 to server, I/O interface128 to peripheral equipment (e.g., camera, speaker, display, etc.), usertracking module 132, user interaction module 133, and other modules 134and sub-modules. The frontend information exchange system 104 furtherhave access to local data 135, which may include local personnel files,local inventory data, etc. These modules utilizes the various real-timedata obtained through various internal and external services, real-timedata received from the peripheral equipment and the backend informationexchange systems, and existing data stored in the local databases, toprovide the in-person interactions with the users and provide productrecommendations to the users.

In accordance with some embodiments, e.g., as shown in FIGS. 1A-1C, theinformation exchange system 100 is implemented to develop deepunderstanding of users (e.g., customers and potential customers) basedon their explicit expressions of their needs, requirements and interestsin products (e.g., through direct interactions with human salesrepresentatives, customer service representatives, and/or the frontendinformation exchange system), their past and present purchase andproduct browsing patterns (e.g., browsing routes, number of repeatedvisits, time length of purchase-decision-making, decision-makingfactors, time of decision, types of products purchased, types ofproducts compared, information sources, solo decision-making orfamily/friends collaborated decision-making, etc.), localexternal/natural conditions (e.g., weather, location, local environment(e.g., prices of electricity, water, availability of alternative energysources, etc.), etc.), external events (e.g., introduction of newproducts and new product features, introduction of new alternativeproducts into the market, introduction of new energy policies andrestrictions by the government, natural disasters, etc.), customersegmentation (e.g., segmentation of customers based on demographic data,such as age, income level, social status, gender, ethnicity, familycomposition, superstition/belief systems, spending habit, profession,trend follower/trend setter, etc.), customer sentiment (e.g., reviews,comments, survey results, etc.), and many other factors.

With the deep understanding of customers in general, as well as the deepunderstanding of specific individual customers, the direct interactionswith each specific customer is formulated in a manner that caters tothat customer's specific needs, style, and preferences, makes thecommunication with that customer more pleasant and the productrecommendations more targeted and timely, and makes the informationexchange more effective to complete sales and encourage continued trustand future visits by the customer.

Furthermore, in some embodiments, the information exchange system 100continuously gathers data, processes data, and mines data to improve theaccuracy of the prediction models and statistics, and decision-makingintelligence. During a particular interaction with a customer, theinformation exchange system 100 utilizes feedback and informationreceived from the individual customer to modify the selection andpriority of the models and decision-making logic used to generate thepredictions, interactions, and recommendations, thereby improving thedata processing speed and efficiency, as well as improving the accuracyand effectiveness of the predictions, interactions, and recommendations.For example, the individual user's reactions, sentiment, and intent(e.g., obtained via analysis of the words, facial expressions, gestures,postures, etc.) are fed back to the information exchange system inreal-time to adding additional parameters to the analysis, prediction,and recommendation models, or reselect the set of models (e.g., removal,replacement, and/or addition of models) that are used to perform theanalysis, prediction, and recommendations, and/or to redirect thedecision-making intelligence/logic, etc. Through continuous engagementwith the customer during the one-on-one on-site interactions, therelevant and pertinent information (e.g., product features, pricepoints, service requirements etc.) is quickly and timely provided to thecustomer, helping the customer to make the purchase decision morequickly and with high confidence.

In some embodiments, the backend information exchange system builds userprofiles and product profiles using various artificial intelligencetechniques. For example, the backend information exchange systemintegrates the knowledge and conclusions from the different data sourcesand analysis methods, such as various machine learning algorithms andspecially engineered decision logic and algorithms, and/or combinationsthereof (e.g., various types of neural networks, deep neural networks,search and optimization analysis, rule-based decision logic,probabilistic methods and fuzzy logic, Bayesian networks, hidden Markovmodels, classifiers and statistical learning models, cybernetics, etc.),to determine the user's identity, key characteristics of the user thatare the most influential to his/her purchasing decision-making, keycharacteristics of the user's current intent, and uses the above toidentify a subset of models and analysis tools to further generate thesuitable responses to the user and provide the most relevantrecommendations using as little computational resources as possible, andas quickly as possible.

In some embodiments, the backend information exchange system providesimage analysis capabilities which process image streams fromstrategically placed cameras at a deployment location of the frontendinformation exchange system, and correlate the results of the imageprocessing to determine the identities and basic characteristics of theusers that visit the deployment location (e.g., through facialrecognition or general demographic analysis based on the appearance ofthe user), their browsing pattern and route throughout the deploymentlocation, and/or whether the users are alone or with family or friends(e.g., by analyzing the distance and cohesiveness of the people in imagestreams of multiple cameras placed in proximity to different productsthroughout of the deployment location of the frontend informationexchange system). In some embodiments, the information exchange systemoptionally identifies a particular visitor or a group of visitors as ahigh potential customer/customer group (e.g., based on the amount oftime that the visitor(s) have stayed in front of a product or a set ofsimilar products, the overall satisfaction scores of the visitor(s))when inspecting a product or a set of similar products, the types ofinformation that the user reviewed for a product or a set of similarproducts, etc.), and provides instructions to adjust the cameras'positioning to follow and/or focus on the high potentialcustomer/customer group for a predefined duration and/or range, in lieuof keeping the fixed camera angle and zoom level for a region in frontof the product. In some embodiments, the information exchange systemalso starts some initial data processing in anticipation of the possiblequestions or requests from the high potential customers that have beenidentified. For example, the information exchange system optionallyretrieves previous purchase records of the high potential customers, andcomputes several high probability decision-making triggers (e.g.,timing, season, price point, deals, product feature, trend, etc.) forthe high-potential customer/customer group before the customer/customergroup makes an initial direct interaction with the frontend informationexchange system. In some embodiments, the information exchange systemprovides an instruction to the frontend information exchange system(e.g., a mobile sales robot) to move to the identified high potentialcustomer/customer group, and initiate direct engagement with thecustomer/customer group. In some embodiments, the frontend informationexchange system optionally provides some initial information orrecommendation to the high potential customer/customer group based onthe initial analysis that is already performed. In some embodiments, thefrontend information exchange system optionally selects a real humansales representative that has been determined to be a best fit (e.g.,based on availability, personality fit, experience level, knowledge ofthe product in which the customer has shown interest, etc.) for the highpotential customer/customer group, and displays an image of the realhuman sales representative on the frontend information exchange system.In some embodiments, speech output of the information exchange systemalso is provided in the voice of the real human sales representative,such that if the customer/customer group eventually wishes to speak to areal human sales representative, there is continuity and sense ofconnection already established for the real human sales representativein the mind(s) of the customer/customer group. In some embodiments, theinformation exchange system retrieves the records of previous humaninteractions with the customer, such as logs made by salesrepresentatives, repair service providers, customer servicerepresentatives, installation service providers, delivery serviceproviders, etc., which include comments and impressions of real humanbeings with the customer to determine characteristics of the interactionstyle, decision-making triggers, preferences, and/or temperament of thecustomer, generates an image of a sales representative for display onthe frontend information exchange system in accordance with thedetermined characteristics, and engages the customer (e.g., askquestions, provide information and suggestions) in a manner that ispredicted to be pleasing and effective for the customer.

In some embodiments, when a customer walks into the store, based onimage processing of the image streams from one or more cameras locatedin the store, the information exchange system (e.g., through facialrecognition, emotion recognition, color recognition, style recognition,accessary recognition, etc.) populate the corresponding retailpurchasing scores calculated using dressing style, gender, age, andhappiness level, etc. For example, i.e., a middle-class professionalmale, a stylish young lady, or a family with children will receivedifferent recommendations including specific features each is likely tochoose (e.g., color choice, size, easiness to operate, etc.).

In some embodiments, for unknown customers, their images, storeengagement, and/or interactions with the frontend information exchangesystem are recorded over many visits and from many sources (e.g.,different deployment locations, social media, public records,self-registration, etc.). The recorded data is used to identify existingcustomer profiles and/or create new unique customer profiles usingvarious specialized artificial intelligence methods. In someembodiments, for existing customers, their profiles can be eitheridentified through facial recognition based on image streams from thecameras installed inside the stores or be identified through customerslogging in at the frontend information exchange system using theirunique identifiers, such as phone numbers, IDs on social networks (e.g.,WeChat, QQ, Weibo, etc.), and/or through conversation andself-introduction during the direction interactions with the frontendinformation exchange system.

In some embodiments, when existing customers' identities are recognized,their historical preferences, shopping behaviors, historical purchasepatterns, customer segmentation, store credits, gift card balance, andcoupon or discount information will be pulled out of the databases inreal-time and used to generate accurate and high probability productrecommendations for each customer based on his unique personas (e.g.,customer profile) already recorded, based on the current in-storeexperience, and/or other third-party public information (e.g., publicrecord regarding social status changes, marital status changes, etc.)

In some embodiments, the information exchange system carries informationincluding current deployment location for each frontend informationexchange system (e.g., store coordinates (e.g., longitude and latitude,street address, city, country, etc.)), store inventory statistics,season of the year, up-coming holiday events, historical sales trends,and/or consumer sentiments that are associated with the currentdeployment location of the frontend information exchange system, etc.This information will also be part of the input for the productrecommendation module to process and identify suitable products forrecommending to each customer that visits the deployment location andengages the frontend information exchange system.

In some embodiments, upon diagnosing for un-matched requirements aftercollecting the customer demands (e.g., if the information exchangesystem fails to locate specific products in the inventory due to lack ofvarieties of products), the information exchange system initiates amulti-tiered recommendation model based on the responses received fromthe customer to provide a supplementary customized recommendation. Forexample, the frontend information exchange system states to the userafter an initial interaction and data processing: “Sorry, we do notcurrently have a double-door fridge in gold color, will you beinterested in a double-door fridge in silver color, or a single doorfridge in gold color?” Customer says, “No, what other colors do you havefor the double-door models?” Based on the above rejection and response,the information exchange system removes or demotes the importance of thecolor parameter relative to the model style (e.g., double-door) for thesubsequent recommendations, and/or selects a different set of predictionmodel(s) where the “double-door” style is used as one of the leadingfactors in the prediction and recommendations (e.g., as oppose to amodel where color and model style are both leading factors). Inaddition, in some embodiments, the personal profile of the customer isalso modified, and the style preference for the “double-door” style isgiven more weight and the color preference is given less weight invarious databases and artificial intelligence models related to thecustomer and/or for the particular kinds of products (e.g., fridge orlarge appliances in general).

In some embodiments, the information exchange system is configured toprovide an augmented reality experience and/or virtual realityexperience (e.g., using various AR/VR equipment available at the currentdeployment location of the frontend information exchange system) thatrelate to the products being requested by the user or recommended to theuser using the product recommendation module. In some embodiments, therequest by a user to try out the AR or VR experience with respect to aparticular product is recorded as a significant trigger event forinitiating the product recommendation module and starts the computationfor the product recommendation before the user initiates a directinteraction with the frontend information exchange system. In someembodiments, the user's reactions (e.g., verbal and facial expressions)to the AR and VR experience are processed and the results are used tomodify the product recommendations and/or the AR and VR experiences. Forexample, if the user initially asked to try a first model of washingmachine with the virtual reality setting, and was not able to figure outhow to properly use the machine (e.g., manipulated multiple buttons andparts of the virtual washing machine with no apparent purpose for morethan a threshold amount of time) and expressed frustration (e.g.,verbally or through facial expressions), the information exchange systemtakes that information as new inputs and generates a new recommendationfor another model with simpler functionalities but similarcharacteristics otherwise (e.g., similar color and dimensions).Alternatively, if the user has a personal profile that suggests that theuser enjoys products with many features in general, the informationexchange system does not suggest a new product, and instead offersassistance regarding the current model to the user. In some embodiments,the frontend information exchange system also generates a presence(e.g., a virtual representation of the frontend information exchangesystem inside of the AR/VR environment, and interacts with the userdirectly within the AR/VR environment). In some embodiments, theinformation exchange system also generates virtual guides (e.g., virtualinstallation personnel) that demonstrate how to use or install theproduct in the AR/VR environment. In some embodiments, the informationexchange system allows the user to visualize multiple home appliancesthat are recommended to the user in their simulated home setup in theAR/VR environment.

In some embodiments, if a customer does not like the productsrecommended by the information exchange system, the information exchangesystem includes artificial intelligence modules that construct aconversation with the customer regarding the customer's reasons forrejecting the recommendations and gather more details about the user'srequirements (e.g., price being too expensive, preference of a differentcolor, other family members requiring special or additionalaccommodations, etc.). In some embodiments, the information exchangesystem adjusts the parameters and/or changes the selection models tolaunch a new round of recommendations according to the new informationthat has been obtained. In some embodiments, a threshold or a set ofcriteria are set to determine when it is time to cease making additionalrecommendations (e.g., when the number of rejected recommendations hasexceeded a preset number, or when the customer's verbal or facialexpression meets a threshold dissatisfaction score, and/or whenparticular predefined keywords or phrases are spoken by the user, etc.)and to redirect the conversation and alleviate the tension (e.g., askingif the user wishes to speak to a human representative, or if the userwishes to receive a coupon, or see a promotional video, or play a gameto win a prize, etc.).

In some embodiments, the frontend information exchange system is mobileand can move and guide the customer to a recommended product that is ondisplay at the deployment location (e.g., the store or exhibition hall).In some embodiments, the frontend information exchange system prints outa map or directions to the recommended product on display. In someembodiments, at the end of the conversation, the frontend informationexchange system presents a summary of the recommendations, and allowsthe user to edit it (e.g., either via a graphical user interfaceprovided by the frontend information exchange system or via verballyinstructions or requests) with additional information and notes; and inthe end, the frontend information exchange system emails or prints outthe final result for the user to take home.

In some embodiments, the information exchange system generatespersonalized discount or promotion deals that are more tailored to eachcustomer based on the customer's interactions with the frontendinformation exchange system, the historical records of the customer(e.g., previous purchases, previous visits, and previous engagementswith the frontend information exchange systems), and/or data from thepresent visit (e.g., browsing route, products inspected, information ordemo seen, sentiment expressed during the visit, etc.).

In some embodiments, the information exchange system utilizes quantityinformation obtained in the present interactions with the customer, andhistoric information obtained about the customer to determine how togenerate quantity recommendations of requested products and relatedproducts, and/or how to generate bundled deals for the customer. Forexample, when the customer requests multiple quantity of a certainproduct, the quantity information and the product category is combinedwith a location change associated with the customer (e.g., a discrepancybetween the deployment location of the frontend information exchangesystem and the previous delivery address of previously purchasedproducts) by the information exchange system to automatically activateand recommend corresponding product bundles that include one or more ofa product from each of multiple related product categories. The productand product categories are identified by the information exchange systemin accordance with the customer profile and the current interactionswith the frontend information exchange system. For example, when acustomer demonstrates an interest in buying multiple quantity of certainproducts (e.g., three small water heaters), a smart situation-matchingmodule is triggered to predict that the customers has one kitchen andtwo bathrooms, and determines the number of air conditioners that thecustomer may need (e.g., four or five air conditioners, one for thekitchen, one for the living room, two or three for the bedrooms thatshare two bathrooms). In some embodiments, if the historic recordindicates that the user has previously purchased one water heater andtwo air conditioners, and no location change has been detected for theuser, the recommended product bundle is adjusted to include four waterheaters and three air conditioners (e.g., the user is estimated to havea kitchen, a living room, four bathrooms, and three bedrooms in hishouse, needing a total of five water heaters and five air conditioners).In some embodiments, the information exchange system also determines howold the existing purchases are, and whether the new request is forreplacement or addition to the existing appliances and the quantityrecommendations are adjusted accordingly. In some embodiments, if acomparison between the current deployment location and the locations inthe historic records indicates that the user has moved to a newlocation, the recommendation is generated based on a new estimate of thenumber and types of rooms that the user has at the new location. Forexample, if the user requests three water heaters, it is estimated thatthe user also needs three or four air conditioners at the new location(e.g., given that it is not customary for people to take theseappliances with them when they move). In some embodiments, if the userrejects the quantity suggestion, a new bundle suggestion is generated bythe information exchange system (e.g., with a higher discount incentive,and/or less financial demand on the customer). In some embodiments, therelated product categories (e.g., the correspondence between productcategories) are pre-stored, and the types of products from the differentproduct categories are also coordinated in style and price point, whichare all selected in accordance with other information already known orpredicted about the user. In some embodiments, the related productcategories are user-specific. In some embodiments, the related productcategories are location-specific. For example, for a deployment locationin a climate that is very hot, the related product categories may be airconditioner and water heaters; while for a deployment location that isin a climate that is very cold, the related product categories may bewater heater and space heaters. In some embodiments, if the informationexchange system detects a status change or location change thatindicates an increased average living cost and spending abilities, theinformation exchange system adjusts the recommendation and/orrecommended bundles accordingly. For example, if the informationexchange system detects that the user has moved from a second-tier cityto a first-tier city or from a less affluent area to a more affluentarea (e.g., based on address changes or store visit location changes),or that the user has changed the clothing style or added expensiveaccessories (e.g., through image processing that detects clothing stylesand accessories, etc.), the information exchange system automaticallyactivates an up-sell model to identify suitable products that meet theuser's present requests but have more expensive features that may beappealing to the user at the present time/status.

In some embodiments, the information exchange system collects andutilizes external data such as weather data, government policy data(e.g., energy policy, water policy, etc.), air pollution data (e.g., PM2.5 measure), etc., when generating customized product recommendationsfor the individual users. For example, the recommendations arecustomized based on current and projected weather conditions,government's up-to-date policies related to air-pollution and energysaving plan, and/or recent natural catastrophes.

In some embodiments, product ratings by other customers, historicalcomments, sentiments, and purchase statistics are provided in additionto product recommendations to the user, to increase their purchasingconfidence for the recommended product and reduce the decision-makingtime.

In some embodiments, the information exchange system takes into accountbelief systems and superstitions when generating product recommendationsto users on different days and at different deployment locations. Thistype of recommendation logic can be applied to both new and existingcustomers. Customers subscribe to, both consciously and subconsciously,beliefs and superstitions regarding when to buy and what to buy. Thesebeliefs and superstitions differ from culture to culture and fromlocation to location. These beliefs and superstitions can lead todifferent recommendations based on Western, Chinese or India beliefs andsuperstitions, and based on different dates, time of day, locations, andpersonal and astrological facts at the time that the recommendations aremade. In some embodiments, with no existing data or input about aparticular customer, recommendations may still be generated for thecustomer based on the favorable factors for the present day and time inaccordance with a predefined belief or fortune-forecasting system (e.g.,astrology, Huangli, Zhouyi, Lunar calendar, etc.). For example, variousrecognized elemental forces, such as metal, wood, earth, air, water,wind and fire, etc. can be favorably associated with (or matched with)different actions (e.g., to buy, repair, inquire, make a deal, etc.) andproduct characteristics (e.g., color, size, material, function, etc.) orproduct categories (e.g., changing air quality, changing airtemperature, changing water temperature, producing fire, producing heat,used in a kitchen, used in a bathroom, used in a bedroom, etc.), and theinformation exchange system, given the characteristics of the presentday/time in accordance with the predefined belief system orfortune-forecasting system, can identify specific products or productcategories (e.g., water heaters or coolers, air conditioners, ovens,etc.) or score them differently in the product recommendation process.The information exchange system also recommends the characteristics(e.g., price, color, size, material, etc.) of products based on thecharacteristics of the present day/time in accordance with thepredefined belief system or fortune-forecasting system. In anotherscenario, if the astrological sign or birth stone, birth animal of thecustomer is known (e.g., derived from a known birthday, or birth month,or age), the information exchange system predicts how the user is doingat the present time (e.g., health, wealth, finance, mood, luck, urge tospend or save, etc.) based on the specific belief system and fortuneforecasting systems relevant to the present customer for a specificperiod (e.g., past and present week or month) and makes correspondingproduct and sale recommendations (e.g., up-sell, vs. offer discounts ordeals, etc.) to the customer. In some embodiments, if the customer'sbelief system is predicted or known, the information exchange systemprovides a suitable description of why the present recommendations wouldbe favorable to the customer in light of the customer's belief systemsand superstitions.

In some embodiments, textual output retrieved from astrological dataservices are parsed for keywords to identify contexts forfinance/wealth, spending vs. saving, elements like air/water/fire/wind,and corresponding product domains and characteristics are activated(like air-conditioner/water cooler/oven/fan) to generate productrecommendations (e.g., products, product categories, productcharacteristics, product features, and coupons and promotion deals,etc.).

Attention is now directed towards FIG. 2 which shows an example userinterface of the frontend information exchange system in an examplescenario in accordance with some embodiments. The example user interfaceis presented on a computer kiosk (e.g., a client device 104). In someembodiments, the conversation can be carried out by a humanoid robot ora mobile device that the user can carry with him/her while he/she browsethrough the deployment location of the frontend information exchangesystem. In addition, although the product recommendation concerns homeappliance recommendations, other types of products may be recommended aswell.

In this example scenario, a customer Mr. Ma entered the store, andbrowsed around for a little while on his own. The in-store camerascaptured his images as he stood in front of some water heater models ondisplay. When his images are processed, the information exchange systemrecognizes Mr. Ma through facial recognition, and starts to retrieverelevant records related to Mr. Ma. The information exchange systemfurther recognizes that Mr. Ma had looked at water heaters recentlyduring another visit with another person and a child at this store.Based on this data, the information exchange system determines that Mr.Ma's visit triggers the enhanced inspection criteria and initiatespreprocessing regarding Mr. Ma's personal profile and productrecommendation, before Mr. Ma makes direct contact with the frontendinformation exchange system. Mr. Ma's previous purchase records, hisprevious interactions with human personnel, locations associated withhis previous purchases, the current weather, his style, price point, andcolor preferences, his decision-making triggers, his demographic data,etc. are processed and the parameters for generating a personalized,real-time product recommendation is generated based on the availabledata. When Mr. Ma approaches the frontend information exchange system(or alternatively, when the frontend information exchange system detectsan opportunity to interact with Mr. Ma), the frontend informationexchange system starts by saying in natural language: “Hello, Mr. Ma,welcome to Shenzhen! I am Xiaomei at the Shenzhen branch store. It seemsthat you are interested in some water heater, is that right?” Thisgreeting is customized for Mr. Ma based on the discrepancy between theregistered deployment location of the frontend information exchangesystem (e.g., Shenzhen), and Mr. Ma's previous delivery address onrecord, so the location of the deployment location is emphasized in thegreeting. A different greeting with a different emphasis or conversationhook would be generated if such a location discrepancy were notdetected. The frontend information exchange system further determinesthat the primary interest of Mr. Ma is water heater because an enhancedinspection event has been detected based on the detection of a repeatvisit regarding the same product or product category recently, and inaddition, the fact that Mr. Ma appeared to be accompanied by familymembers during at least one of these visits. In some embodiments, animage of the sales representative (e.g., Xiaomei) that previously hadgood rapport with Mr. Ma is displayed, and an interaction style (e.g.,casual vs. formal, respectful vs. relatable, succinct vs. elaborative,etc.) and voice characteristics of that sales representative are used togenerate the speech output of Xiaomei. In response to hearing Xiaomei'sgreetings, Mr. Ma was forthcoming with additional information, and says“Yes. I want to buy three water heaters.” The frontend informationexchange system, extracts the quantity keyword from Mr. Ma's answer, andactivates a cross-sell and upsell model that utilizes that quantityinformation. The cross-sell and upsell model further utilizes thelocation change information to determine that the quantity of relatedappliances that should be recommended to Mr. Ma. For example, in apre-stored related product categories database, the water heater belongsto a first category, and has a first related category of airconditioners and a second related category of ovens. The air conditionercategory is associated with a higher profit margin than the ovencategory, and a higher cross-sell quantity correspondence with the waterheater category (e.g., usually a household only needs one ovenregardless of how many water heaters it needs, and usually a householdneeds at least as many air conditioners as water heaters). Based on adetermination that there is a location discrepancy between the currentdeployment location of the frontend information exchange system and theuser's previous delivery location, the information exchange systemutilizes a first cross-sell quantity matching model to estimate thenumber of air conditioners that should be recommended to Mr. Ma (e.g., amodel that does not take into account of the number of previouslypurchased air conditioners for the previous location of Mr. Ma). If alocation discrepancy had not been detected, the information exchangesystem would utilize a different cross-sell quantity machine model toestimate the number of air conditioners that should be recommended toMr. Ma (e.g., a model that takes into account of previously purchasedappliances and their quantities, and the appliances age and model, todetermine the new quantity for air conditioners that should berecommended to Mr. Ma's new location). Based on purchase decisiontriggers known about Mr. Ma, the information exchange system alsodetermines that value as compared to styles are more important factorsfor Mr. Ma's purchasing decision making. The frontend informationexchange system proceeds to generated recommendations based on the aboveprocessing models and logic, and outputs the following cross-sell andupsell recommendations “Ok. Would you like to also look at some airconditioners? We currently have some great deals for air conditioner andwater heater bundles.” When Mr. Ma consents to seeing somerecommendations, the frontend information exchange system provides therecommended bundle deals to Mr. Ma, and also informs Mr. Ma that thefeatures of the recommended models are suitable for young professionalswith families (based on the knowledge that Mr. shopped with anotheradult and a child, and other demographic information that are deducedfrom the appearance and past purchasing habit of Mr. Ma). In theinformation provided to Mr. Ma, the model numbers, quantities, and pricediscounts are included for each recommended bundle. In addition, theimages that are presented shows color and styles selections that areselected based on previous purchases made by Mr. Ma. The frontendinformation exchange system also informs Mr. Ma how the style and colorsare selected, and offers the option to change them if requested. In someembodiments, if Mr. Ma does makes a request to change the style or coloror other characteristics that are selected based on an existingrecommendation strategy used by the recommendation generation engine,the information exchange system utilizes the rejection and newrequirement in selecting a different set of models and/or a differentpriority of the models in generating new recommendations for similarsituations in the future, in addition to revising the recommendationsprovided at the present time. In this example, Mr. Ma chose one of therecommended bundles, and the information exchange system offers tomodify the delivery address on record. Once Mr. Ma confirms that thelocation discrepancy is actually a change in home address, theinformation exchange system triggers the upsell and cross-sell moduleagain to inquire about other appliance needs of the customer at the newlocation. The above is merely an example scenario to illustrate therecommendation generation process for providing real-time, personalizedproduct recommendations. Other features are apparent based on otherdisclosures and details provided herein.

FIG. 3 is a flowchart diagram of a method 300 of providing productrecommendations in accordance with some embodiments. In someembodiments, the method 300 is performed by an information exchangesystem (e.g., a backend server 106, in conjunction with one or morefrontend client devices 104). In some embodiments, method 300 isgoverned by instructions that are stored in a non-transitory computerreadable storage medium and the instructions are executed by one or moreprocessors of the client and server systems.

In some embodiments, a method of providing machine-generated productrecommendations, includes: at an electronic device (e.g., a backendserver hosting a product recommendation engine) having one or moreprocessors, and memory: deploying (302) a frontend information exchangesystem (e.g., a computer kiosk or sales robot), wherein the frontendinformation exchange system provides an input user interface configuredto detect one or more in-person close-proximity interactions between auser and the frontend information exchange system (e.g., interactions byvoice, touch, sight, gesture, etc. that are enabled by the closephysical proximity between the user and the frontend informationexchange system); registering (304) a current deployment location of thefrontend information exchange system (e.g., the location registrationcan be part of the set-up process or initialization process of thecomputer kiosk or sales robot). In some embodiments, the frontendinformation exchange system includes a self-location system thatautomatically transmits its current deployment location to the backendserver when the frontend information exchange system is activated. Themethod 300 further includes: detecting (306) a first in-personclose-proximity interaction between a first user and the frontendinformation exchange system at the current deployment location of thefrontend information exchange system, wherein the first in-personclose-proximity interaction specifying a first product category forwhich the first user is seeking product recommendations. For example,when a user directly interacts with the sales robot or walks up to thecomputer kiosk, and a camera at the location of the sales robot or kioskcaptures an image of the user that meet predefined criteria (e.g.,predefined proximity threshold), or a speech input is captured from theuser requesting assistance, or a user scans or types in a user ID, etc.,the start of a first in-person close-proximity interaction is detected,and the frontend information exchange system notifies the backend serverof an in-person close proximity interaction with the user is initiated.In some embodiments, the first in-person close proximity interactionspecifies a first product category for which the user is seeking productrecommendation. For example, the user selects the product categorydirectly by speaking to the frontend information exchange system “I amlooking for some water heaters.” In some embodiments, the first productcategory is derived from the user's browsing patterns at the deploymentlocation and the time that the user has spent in an area associated withthe first product category. In some embodiments, in response to thefirst in-person close-proximity interaction between the first user andthe frontend information exchange system, the information exchangesystem automatically generates (310) a first product recommendation,wherein the first product recommendation includes a first product of thefirst product category (e.g., a product that is selected based at leastin part on the registered current deployment location of the frontendinformation exchange system and other characteristics known about theuser and/or about the present market). For example, if the user wants tolook at water heaters, the recommendation includes a first water heatermodel that is selected based on the sales volume for various models andthe estimated household size of the user. The information exchangesystem detects (312) a respective prior purchase record of the firstuser for a second product in a second product category that isassociated with the first product category. For example, the informationexchange system stores pre-established associations between productcategories based on various cross-sell strategies (e.g., based onclimate conditions, based on capacity, based on location of use, basedon energy efficiency levels, etc.). For example, based on capacity,whole house water heaters and whole house central heating systems arerelated categories, and individual water heaters and room-based airconditioners are related categories; based on energy efficiency levels,high efficiency laundry machines and high efficiency dishwashers arerelated categories; based on usage locations, water heaters areassociated with both kitchen appliances (e.g., oven, fridge, microwave,dishwasher, etc.) and bathroom appliances (e.g., bath fixtures, bidets,etc.), etc. In accordance with a determination that the respective priorpurchase record for the second product indicates a purchase or deliverylocation outside of a predefined geographic region of the registeredcurrent deployment location of the information exchange system (e.g.,when the frontend information exchange system is deployed in Shenzhen,and the previous delivery location for an oven purchased by the user isin Beijing, and the two cities are outside of normal everyday traveldistance of a user with a fixed home address), the information exchangesystem automatically augments (314) the first product recommendation toinclude a third product from the second product category that isselected based on characteristics of the second product (for example,the information exchange system identifies an air conditioner from therelated category (e.g., the room-based air conditioner category which isrelated to the individual water heater category) and adds it to thewater heater recommendation, e.g., as a bundle). In accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of theinformation exchange system, the information exchange systemautomatically refines (316) the first product recommendation to furtherdefine one or more characteristics of the first product based on thecharacteristics of the second product. For example, if the deploymentlocation and the user's previous delivery location are both in Shenzhen,the information exchange system uses the previous purchase record todetermine one or more parameters that can be used to further refine therecommendation for the water heater. For example, if the user haspreviously purchased an air conditioner with a metallic finish, and amodern style, the recommendation for the water heater can have the sameor similar modern style and metallic finish. If the previous purchaseincludes three sets of bath fixtures, the water heater models areselected to in accordance with compatibility to the bath fixtures andthe quantity recommended is selected based on the quantity of the bathfixtures (e.g., at least three or four for three bathrooms and akitchen). Some of the information from the previous purchases (e.g.,style preferences) are still utilized in refining the recommendation ofthe first product when the location discrepancy were detected, whileother information from the previous purchase (e.g., the quantityinformation) is only taken into consideration if there is no locationdiscrepancy detected. In some embodiments, depending on whether there isa location discrepancy detected, some of the previous purchaseinformation is used differently. For example, when there is no locationdiscrepancy detected, the previous purchase quantity information is usedto estimate the number of rooms in the house directly, and the number ofrooms is used to select the quantity for the current productrecommendation; and when there is a location discrepancy detected, theprevious purchase quantity information is used to estimate the householdsize, and the number of current product recommendation may be selectedwithout consideration of household size, but the model of the currentproduct recommendation may be selected based on household size. Theinformation exchange provides (318), through the frontend informationexchange system, the first product recommendation to the first userafter the automatic augmenting or refining of the first productrecommendation. This is for example illustrated in FIG. 2.

In some embodiments, in response to the first in-person close-proximityinteraction between the first user and the frontend information exchangesystem: the information exchange system detects a respective priorpurchase record of the first user for the first product (e.g., user haspreviously purchased one or more water heaters and is now requestingwater heater again) in addition to the respective prior purchase recordfor the second product (e.g., previous purchase of one or more airconditioners); and in accordance with a determination that therespective prior purchase record for the first product (e.g., theprevious purchase of water heater(s)) indicates that same purchase ordelivery location as the respective prior purchase record for the secondproduct (e.g., previous purchase of the air conditioner(s)), theinformation exchange system automatically augments the first productrecommendation to include the second product from the second productcategory (e.g., a recommendation of the air conditioner that waspreviously purchased), irrespective of whether the purchase or deliverylocation indicated in the respective prior purchase records for thefirst and second products is within the predefined geographic region ofthe registered current deployment location of the information exchangesystem. For example, if the user has previously bought one airconditioner that is still a current model on sale now or a similar modelis on sale now, the information exchange system recommends that airconditioner model or the similar model to the user in addition to therecommendation of water heater that has been requested. For example,when the information exchange system determines that there is adiscrepancy between the number of water heater requested and the numberof air conditioners that have been purchased already (note, the quantitycorrespondence for water heater and air conditioner is differentdepending on whether the user has changed his home location), theinformation exchange system recommends the user to purchase more of theair conditioners to remove that quantity discrepancy.

In some embodiments, the first product recommendation is provided to thefirst user after the automatic augmenting of the first productrecommendation to include the third product from the second productcategory that is selected based on the characteristics of the secondproduct. In some embodiments, the information exchange system detects,through the frontend information exchange system, a user input rejectingthe third product from the second product category that is included inthe first product recommendation; and in response to detecting the userinput rejecting the third product from the second product category thatis included in the first product recommendation, revising the firstproduct recommendation to remove the third product from the secondproduct category and include a fourth product from a respective productcategory for which the first user has no prior purchase record. Forexample, when rejecting the previously purchased air conditionerincluded in the first recommendation, the user may say “I don't want anyair conditioners now” or “I don't like that air conditioner”, or “I wanta different colored air conditioner this time”, or “That air conditionerwas too noisy”, etc. The information exchange system analyzes the user'sintent regarding the recommendation for the previously purchasedproduct, and if the user does not wish to purchase any of that productat the present time, the information exchange system removes therecommendation for the previously purchased air conditioner, andrecommends something completely different (e.g., a dish washer, if theinformation exchange system determines that the user has not previouslypurchased a dish washer in accordance with the past record). In someembodiments, the recommendation for the air conditioner may not beexactly the same model as the one previously purchased, and instead, itmay be a model that is selected based on the previously purchased modeland that has parameters fitting the present time and characteristics ofthe user. In such a case, the information exchange system may alsorevise the recommendation and choose a different model from the sameproduct category based on new information contained in the user'sresponse.

In some embodiments, the first product recommendation is provided to thefirst user after the automatic refining of the first productrecommendation to further define one or more characteristics of thefirst product based on the characteristics of the second product, andwherein the automatic refining includes: determining a size category ofthe first product (e.g., size category of water heater is medium orsmall); determining a size category of the second product (e.g., thesize category of a room-based air conditioner is medium, the sizecategory of a fridge is large); in accordance with a determination thatthe size category of the first product is distinct from the sizecategory of the second product (e.g., the size categories of the twoproducts are different, one large, one small), defining the one or morecharacteristics of the first product based on the characteristics of thesecond product in accordance with a pre-stored contrasting matchingscheme (e.g., colors and/or textures that are high contrast for the twodifferent size categories); and in accordance with a determination thatthe size category of the first product is the same as the size categoryof the second product (e.g., both are large size categories), definingthe one or more characteristics of the first product based on thecharacteristics of the second product in accordance with a pre-storedharmonizing matching scheme (e.g., the same or similar finishes and/orcolors for the two large appliances).

In some embodiments, the information exchange system establishes acorrespondence database that includes, for each respective productcategory of a plurality of product categories, a corresponding set offortune forecasting parameters that are determined to be compatible withthe respective product category (e.g., for air conditioners, thecompatible parameters include the elemental forces ice, metal, air,water, and incompatible parameters include the elemental forces fire,earth, wood, etc.; while for ovens the compatible parameters include theelemental forces fire, metal, and air; and the incompatible elementalforces wood, water, and earth, etc.). In response to the first in-personclose-proximity interaction between the first user and the frontendinformation exchange system: the information exchange system identifiesa set of fortune forecasting parameters that are currently valid for thefirst user (e.g., the fortune forecasting principles' of the user'sbelief system, or a third-party fortune-telling website predicts thatthe people of the user's birth month have favorable outcomes when takingactions involving the elemental force of water and/or have unfavorableoutcomes when taking actions involving the elemental force of fire). Inaccordance with the correspondence database, the information exchangesystem identifies a third product category from the plurality of productcategories that corresponds to the set of fortune forecasting parametersthat are currently valid for the first user. For example, given that thefortune forecasting indicates that the user should engage in activityinvolving the elemental force of water, the information exchange systemrecommends an air conditioner as opposed to an oven to the user, even ifboth are permissible recommendations based on other considerations. Theinformation exchange system augments the first product recommendation toinclude one or more products from the third product category with thefirst product.

In some embodiments, the first in-person close-proximity interactionspecifies a first product quantity for the first product category thatthe first user intends to purchase. The information exchange systemautomatically generates a digital deal that bundles the first product ofthe first product quantity and the third product of a second productquantity that is selected based on the first product quantity. Anexample of the quantity correspondence is described with respect to theexample shown in FIG. 2. Other quantity correspondences are alsooptionally used in different embodiments and scenarios. In someembodiments, selecting the second product quantity includes: inaccordance with the determination that the respective prior purchaserecord for the second product indicates a purchase or delivery locationoutside of the predefined geographic region of the registered currentdeployment location of the information exchange system, selecting thesecond product quantity based on the first product quantity withoutregard to a prior purchase quantity of the second product indicated inthe respective prior purchase record; and in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of theinformation exchange system, selecting the second product quantity basedon the first product quantity and the prior purchase quantity of thesecond product indicated in the respective prior purchase record. Forexample, the quantity is adjusted depending on whether the user hasmoved to a new home. The quantity is recalculated without or withoutregard to the previous purchase quantities, depending on whether theuser has moved, in some embodiments.

It should be understood that the particular order in which theoperations have been described is merely exemplary and is not intendedto indicate that the described order is the only order in which theoperations could be performed. One of ordinary skill in the art wouldrecognize various ways to reorder the operations described herein.Additionally, it should be noted that details of other processesdescribed herein with respect to other methods and/or processesdescribed herein are also applicable in an analogous manner to method300 described above.

In some embodiments, the method of providing machine-generated productrecommendations includes deploying a frontend information exchangesystem, wherein the frontend information exchange system provides aninput user interface configured to detect one or more in-personclose-proximity interactions between a user and the frontend informationexchange system (e.g., interactions by voice, touch, sight, gesture,etc. that are enabled by the close physical proximity between the userand the frontend information exchange system). The information exchangesystem activates respective input streams (e.g., image streams orperiodic samples from the image streams) from one or more on-sitecameras located at a current deployment location (e.g., a brick andmortar store or exhibition hall) of the frontend information exchangesystem, wherein each of the one or more on-site cameras is located inproximity to a respective sample product on display at the currentdeployment location of the frontend information exchange system (e.g.,each camera is assigned to a region of the store or exhibition hallwhere a samples of a particular product or product category are ondisplay and captures images of people visiting that region of the storeor exhibition hall). In accordance with the respective input stream of afirst camera of the one or more on-site cameras, the informationexchange system registers a first inspection event of a first user inassociation with a first sample product on display at the currentdeployment location of the frontend information exchange system (e.g.,an inspection event is recognized by the information exchange systembased on image processing of the images). For example, a camera pointedat the region where water heaters are on display captures images of auser that has stopped in that region. The images are processed andthrough facial recognition and motion analysis, the information exchangesystem determines that the user is in fact looking at a particularsample product on display in that region, and the user's identity andsome of the user's demographic characteristics can be determined basedon the user's facial features and appearance in general. In someembodiments, the user's facial expressions and voices may also byanalyzed to determine the emotional state of the user and comments hemade during his inspection of the products in that region. In someembodiments, the duration of the inspection for each product in thatregion is determined based on the number of frames that included theuser's images. In some embodiments, whether the user is in a group ofmultiple users (e.g., a family or a group of friends) is also determinedbased on image analysis (e.g., by clustering the people in the images,and analyzing the overall cohesiveness (e.g., measured by variations indistances between the people in the various clusters) of the people indifferent clusters across the images from multiple cameras at differentlocations). In some embodiments, the sample product that is beinginspected is optionally identified based on the location associated witha respective camera and/or image processing of the images from therespective camera (e.g., appearance of the product, IDs, barcodes, orother key visually identifying characteristics). The informationexchange system detects a first in-person close-proximity interactionbetween the first user and the frontend information exchange system atthe current deployment location of the frontend information exchangesystem. For example, when a user directly interacts with the sales robotor walks up to the computer kiosk, and a camera at the location of thesales robot or kiosk captures an image of the user that meet predefinedcriteria (e.g., predefined proximity threshold), or a speech input iscaptured from the user requesting assistance, or a user scans or typesin a user ID, etc., the start of a first in-person close-proximityinteraction is detected, and the frontend information exchange systemnotifies the backend server of an in-person close proximity interactionwith the user is initiated. In response to the first in-personclose-proximity interaction between the first user and the frontendinformation exchange system, the information exchange systemautomatically generates a first product recommendation, including: inaccordance with a determination that the first inspection event of thefirst user that has been registered in association with the first sampleproduct meets enhanced inspection criteria, wherein the enhancedinspection criteria includes a criterion that is met when at least asecond inspection event of the first user exists in a plurality ofpreviously stored inspection events associated with the respective firstsample product, automatically adding a product-specific description ofthe first sample product in the first product recommendation; and inaccordance with a determination that the first inspection event of thefirst user that has been registered in association with the first sampleproduct does not meet the enhanced inspection criteria, forgoingincluding the product-specific description of the first sample productin the first product recommendation. For example, if it has beendetermined that the same user has previously inspected the same sampleproduct, it is more likely that the user is actually interested in theinspected sample product, and the enhanced inspection criteria are met.Therefore, the level of specificity of the product recommendation isincreased to a level where the product-specific description of theinspected product is mentioned in the recommendation, such as the name,model number, unique features, etc. of the inspected product, relativeto a more generic mention or no mention of other products. On the otherhand, if the enhanced inspection criteria are not met, it is more likelythat the user is still in the general browsing stage of hisdecision-making. Therefore, the level of specificity of the productrecommendation is kept at a relatively low level, and the recommendationis more at a stage of exploration of the user's interest and needs. Sodescriptions of power efficiency, color, style, general features, etc.are included in the recommendations to cover a range of possible choicesof products, rather than a specific product. A listing of multipleproducts that are recommended user without special emphasis on aparticular product is not product-specific, even if list includes foreach item some product-specific description such as model number, size,etc. In some embodiments, multiple levels of specificity of productdescription in the recommendation are respectively correlated todifferent ranges of scores assigned to the current inspection event. Thescore assigned to the current inspection event changes as more facts areaccumulated and discovered in relation to the current inspection event.For example, previous inspection event by the same person of the sameproduct increases the score. The fact that inspection event isregistered when the user is with multiple people that travels of asgroup (e.g., family or friends) also increases the score. In anotherexample, a request to interact with the product in a virtual reality oraugmented reality environment also increases the score. Many otherfactors can be taken into consideration to increase or decrease thescore, and to trigger or not trigger the satisfaction of the enhancedinspection criteria. The factors are not exhaustively enumerated hereinin the interest of brevity. The information exchange system provides,through the frontend information exchange system, the first productrecommendation to the first user.

In some embodiments, for each of a plurality of sample products ondisplay at the current deployment location of the frontend informationexchange system, the information exchange system determines a respectiveinspection time that the first user is within a respective inspectionzone of said each sample product in accordance with the respective inputstream of one or more on-site cameras located in proximity to said eachsample product. The information exchange system selects, from theplurality of sample products, a sample product for which the first userhas the longest inspection time as the first sample product. Forexample, the information exchange system prioritizes the task of goingback to the database to search for previous events by selecting theinspection event that is most likely to indicate user interest as thefirst sample product to perform the evaluation regarding whetherenhanced inspection criteria are met. By doing such prioritization, therecommendations can be prepared more promptly with less expenditure ofcomputing resources.

In some embodiments, for each of a plurality of sample products ondisplay at the current deployment location of the frontend informationexchange system, the information exchange system determines whether thefirst user is within a respective inspection zone of said each sampleproduct concurrently with one or more other users as a first recognizedgroup. From the plurality of sample products, the information exchangesystem selects, as the first sample product, a respective sample productof which the first user is found to be concurrently present within therespective inspection zone with the one or more other users as the firstrecognized group. For example, the information exchange systemprioritizes the task of going back to the database to search forprevious events by selecting the inspection event that is most likely toindicate user interest as the first sample product to perform theevaluation regarding whether enhanced inspection criteria are met. Bydoing such prioritization, the recommendations can be prepared morepromptly with less expenditure of computing resources. In someembodiments, for each of a plurality of sample products of which thefirst user is found to be concurrently present within the respectiveinspection zone with one or more other users as the first recognizedgroup, the information exchange system calculates a respectiveaggregated satisfaction score for the first user and the one or moreother users in the first recognized group based on analysis of therespective input streams of the one or more on-site cameras. Forexample, the facial expressions of the users are analyzed to determinetheir mood and happiness level. In general, if the group as a whole isin a good mood and satisfaction scores are above a threshold, theproduct recommendation is more likely to succeed with respect to thefirst sample product or any product in general. The information exchangesystem selects, from the plurality of sample products of which the firstuser is found to be concurrently present within the respectiveinspection zone, a sample product for which the first user and the oneor more other users in the first recognized group has the highestaggregated satisfaction score, as the first sample product. For example,the information exchange system prioritizes the task of going back tothe database to search for previous events by selecting the inspectionevent that is most likely to indicate user satisfaction as the firstsample product to perform the evaluation regarding whether enhancedinspection criteria are met. By doing such prioritization, therecommendations can be prepared more promptly with less expenditure ofcomputing resources.

In some embodiments, providing, through the frontend informationexchange system, the first product recommendation to the first userincludes providing at least one natural language statement that includesone or more key words each describing a respective factor used togenerate the first product recommendation. For example, the frontendinformation exchange system may output a speech input like this “Mr. Ma,it seems you are interested in our AXP for water heater. This model isparticularly popular among young professionals with small families, suchas yourself because it is highly energy efficient and can be controlledvia smart devices such as an Android device (e.g., assuming theinformation exchange system determined that the user is wearing anAndroid watch through image analysis). We also have a similar model AXZthat has an enhanced self-regulation program in addition to the featuresof the AXP model in case you are interested.” In the above statement,the keyword phrases “young professional” “small family” “energyefficient” and “controllable by Android device” are all factors used togenerate the recommendation of the water heater AXP model. The aboverecommendation does include product-specific descriptions such as themodel numbers AXP, and the product-specific feature “self-regulationprogram” for the AXZ model. A corresponding example of more generalrecommendation that does not include the product-specific descriptionwould be “Mr. Ma, it seems that you are interested in water heaters. Thefollowing models are very popular among young professionals with smallfamilies, such as yourself because it is highly energy efficient and canbe controlled via smart devices such as an Android device (List themodels: . . . ). Note that although the model numbers and model specificdescriptions may be included as part of the list. The list is notproduct-specific to the first product that was inspected by the user.

In some embodiments, the information exchange system determines a set ofselection factors that are relevant to the first user. The informationexchange system identifies two or more alternative models for generatingproduct recommendations for the first user based on the set of selectionfactors that are determined to be relevant to the first user, whereinthe first product recommendation is generated based on a first model ofthe two or more alternative models. After providing the first productrecommendation to the first user, the information exchange systemdetects a second in-person close-proximity interaction between the firstuser and the frontend information exchange system at the currentdeployment location of the frontend information exchange system, whereinthe second in-person close-proximity includes a rejection of the firstproduct recommendation. In accordance with a determination that thesecond in-person close-proximity interaction between the first user andthe frontend information exchange system includes a rejection of thefirst product recommendation, the information exchange systemautomatically generates a second product recommendation using a secondmodel of the two or more alternative models based on the set ofselection factors that are determined to be relevant to the first user.By switching the models used by the information exchange system after arejection, the underlying assumptions of the recommendations can beupdated completely, and the set of selection factors used, theirrelative priority in the model, and their relative importance in themodel, and their relative usage in the decision logic of the model canall be updated without restriction, thus generating a betterrecommendation based on the new information obtained through theinteraction (e.g., the reasons for rejection, the style of therejection, or the mere rejection itself). This is different from merelyadjusting the parameters or weights used within a fixed model, becausethe underlying assumptions of the model may be wrong, or the model isnot optimized for the current situation (e.g., the situation asdescribed by all the facts and parameters currently known). In someembodiments, in accordance with a determination that the secondin-person close-proximity interaction between the first user and thefrontend information exchange system includes a rejection of the firstproduct recommendation, the information exchange system analyzes thesecond in-person close-proximity interaction to determine one or moreadditional factors that are relevant to the first user that modifies oradds to the set of selection factors. For example, if the user says “Idon't like my fridge to be the same color as my oven” or “I seem to haveseen a report on leakage issue for this type of water heaters”, or “Iwould like to wait on the water heater purchase until the end of themonth”, etc. Each of these rejections can be processed by theinformation exchange system to extract factors that are relevant to theuser's decision. For example, based on the first statement, the color ofthe oven and a color difference requirement can be the additionalfactors identified by the information exchange system. For the secondstatement, product quality, explanation of negative publicity, availablewarranty options, and improvements made on a known issue can be theadditional factors identified by the information exchange system. Forthe third statement, timing, price point, fortune-forecastingconsiderations, other types of products the user is currentlyconsidering may be the additional factors identified by the informationexchange system. The analysis of the information exchange system isoptionally based on keywords extraction, natural language processing toanalyze intent, rule-based, or other artificial intelligence techniques.In some embodiments, if multiple additional factors are identified, theinformation exchange system optionally prioritizes the additionalfactors based on the above analysis as well. The information exchangesystem then automatically generates the second product recommendationusing the second model based on the one or more additional factors incombination with the set of selection factors that are determined to berelevant to the first user.

It should be understood that the particular order in which theoperations in FIG. 3 have been described is merely exemplary and is notintended to indicate that the described order is the only order in whichthe operations could be performed. One of ordinary skill in the artwould recognize various ways to reorder the operations described herein.Additionally, it should be noted that details of other processesdescribed herein with respect to other methods and/or processesdescribed herein are also applicable in an analogous manner to method300 described above.

FIG. 4 is a block diagram illustrating a representative server 108(e.g., serving as a backend information exchange system) in accordancewith some embodiments. Server 108, typically, includes one or moreprocessing units (CPUs) 402 (e.g., processors 112 in FIG. 1A), one ormore network interfaces 404, memory 406, and one or more communicationbuses 408 for interconnecting these components (sometimes called achipset). Server 108 also optionally includes a user interface 410. Userinterface 410 includes one or more output devices 412 that enablepresentation of media content, including one or more speakers and/or oneor more visual displays. User interface 410 also includes one or moreinput devices 414, including user interface components that facilitateuser input such as a keyboard, a mouse, a voice-command input unit ormicrophone, a touch screen display, a touch-sensitive input pad, agesture capturing camera, or other input buttons or controls. Memory 406includes high-speed random access memory, such as DRAM, SRAM, DDR RAM,or other random access solid-state memory devices; and, optionally,includes non-volatile memory, such as one or more magnetic disk storagedevices, one or more optical disk storage devices, one or more flashmemory devices, or one or more other non-volatile solid-state storagedevices. Memory 406, optionally, includes one or more storage devicesremotely located from one or more processing units 402. Memory 406, oralternatively the non-volatile memory within memory 406, includes anon-transitory computer readable storage medium. In someimplementations, memory 406, or the non-transitory computer readablestorage medium of memory 406, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   operating system 416 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   network communication module 418 for connecting server 108 to        other computing devices (e.g., client devices 104 or third-party        services 122) connected to one or more networks 110 via one or        more network interfaces 404 (wired or wireless);    -   presentation module 420 for enabling presentation of information        (e.g., a user interface for application(s), widgets, web pages,        audio and/or video content, text, etc.) at server 108 via one or        more output devices 412 (e.g., displays, speakers, etc.)        associated with user interface 410;    -   input processing module 422 for detecting one or more user        inputs or interactions from one of the one or more input devices        414 and interpreting the detected input or interaction;    -   one or more applications 424 for execution by server 108;    -   server-side module 106, which provides server-side data        processing and functionalities, including but not limited to:        -   ecommerce data processing module 152 for processing market            data, sales data, product data, competitor data, etc. to            identify aggregated data related to price, market segments,            product segments, industry trends, product features,            consumer sentiment, product review, cross-sell correlation,            etc.;        -   purchase/delivery/service history processing module 153 for            processing past records of users' purchase, delivery and            service history records to determine home address, social            class, product purchased, time of purchase of product,            factors influencing purchase decisions, satisfaction, past            experiences with the product, knowledge of product features,            style preferences, price point, etc.;        -   Historic human interaction log processing module 154 for            processing past records of users' interactions with human            personnel as documented by the human personnel to determine            relevant characteristics of the individual users, such as            personal style, human interaction style, living environment,            demographic data, personal tidbits for developing rapport,            etc.;        -   Current in-person interaction module 156 for processing            input from the current visit of the user such as images,            speech input, and other relevant inputs and peripheral data            at the present time (e.g., current weather, current external            events, etc.).        -   Data processing integration module 158 for selecting the            suitable processing modules, prioritizing their operations,            and integrating results from some processing modules to            provide input to other processing modules, based on the            current available data, newly acquired data, and            intermediate results;        -   Augmented reality and virtual reality processing module for            generating augmented reality and virtual reality experiences            for the user based on the products that the user has            inspected, products recommended to the user, products the            user requests, and the user's characteristics, preferences,            interaction styles, etc.        -   Product recommendation module 162 for generating product            recommendations to the user based on the results of the            different modules;        -   other modules for performing other functions set forth            herein.

Each of the above-identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 406, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 806,optionally, stores additional modules and data structures not describedabove.

In some embodiments, at least some of the functions of server system 108are performed by client device 104, and the corresponding sub-modules ofthese functions may be located within client device 104 rather thanserver system 108. In some embodiments, at least some of the functionsof client device 104 are performed by server system 108, and thecorresponding sub-modules of these functions may be located withinserver system 108 rather than client device 104. Client device 104 andserver system 108 shown in the Figures are merely illustrative, anddifferent configurations of the modules for implementing the functionsdescribed herein are possible in various embodiments.

While particular embodiments are described above, it will be understoodit is not intended to limit the application to these particularembodiments. On the contrary, the application includes alternatives,modifications and equivalents that are within the spirit and scope ofthe appended claims. Numerous specific details are set forth in order toprovide a thorough understanding of the subject matter presented herein.But it will be apparent to one of ordinary skill in the art that thesubject matter may be practiced without these specific details. In otherinstances, well-known methods, procedures, components, and circuits havenot been described in detail so as not to unnecessarily obscure aspectsof the embodiments.

FIG. 5 is a block diagram illustrating a representative client device104 (e.g., serving as a frontend information exchange system) inaccordance with some embodiments. Client device 104, typically, includesone or more processing units (CPUs) 502 (e.g., processors 128), one ormore network interfaces 504, memory 506, and one or more communicationbuses 508 for interconnecting these components (sometimes called achipset). Client device 104 also includes a user interface 510. Userinterface 510 includes one or more output devices 512 that enablepresentation of media content, including one or more speakers and/or oneor more visual displays. User interface 510 also includes one or moreinput devices 514, including user interface components that facilitateuser input such as a keyboard, a mouse, a voice-command input unit ormicrophone, a touch screen display, a touch-sensitive input pad, agesture capturing camera, or other input buttons or controls.Furthermore, some client devices 104 use a microphone and voicerecognition or a camera and gesture recognition to supplement or replacethe keyboard. In some embodiments, client device 104 further includessensors, which provide context information as to the current state ofclient device 104 or the environmental conditions associated with clientdevice 104. Sensors include but are not limited to one or moremicrophones, one or more cameras, an ambient light sensor, one or moreaccelerometers, one or more gyroscopes, a GPS positioning system, aBluetooth or BLE system, a temperature sensor, one or more motionsensors, one or more biological sensors (e.g., a galvanic skinresistance sensor, a pulse oximeter, and the like), and other sensors.Memory 506 includes high-speed random access memory, such as DRAM, SRAM,DDR RAM, or other random access solid-state memory devices; and,optionally, includes non-volatile memory, such as one or more magneticdisk storage devices, one or more optical disk storage devices, one ormore flash memory devices, or one or more other non-volatile solid-statestorage devices. Memory 506, optionally, includes one or more storagedevices remotely located from one or more processing units 502. Memory506, or alternatively the non-volatile memory within memory 506,includes a non-transitory computer readable storage medium. In someimplementations, memory 506, or the non-transitory computer readablestorage medium of memory 506, stores the following programs, modules,and data structures, or a subset or superset thereof:

-   -   operating system 516 including procedures for handling various        basic system services and for performing hardware dependent        tasks;    -   network communication module 518 for connecting client device        104 to other computing devices (e.g., server system 108)        connected to one or more networks 110 via one or more network        interfaces 504 (wired or wireless);    -   presentation module 520 for enabling presentation of information        (e.g., a user interface for presenting text, images, video,        webpages, audio, etc.) at client device 104 via one or more        output devices 812 (e.g., displays, speakers, etc.) associated        with user interface 510;    -   input processing module 522 for detecting one or more user        inputs or interactions from one of the one or more input devices        514 and interpreting the detected input or interaction;    -   one or more applications 524 for execution by client device 104        (e.g., payment platforms, media player, and/or other web or        non-web based applications);    -   client-side module 102, which provides client-side data        processing and functionalities, including but not limited to:        -   user tracking module 132 for tracking movement of the user            at the deployment location of the client device;        -   user interaction module 133 for implementing the direct and            indirect interactions with the user during the user's visit            at the deployment location; and        -   other modules 134 for performing other functions set forth            herein.

Each of the above-identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 506, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 806,optionally, stores additional modules and data structures not describedabove.

In some embodiments, at least some of the functions of server system 108are performed by client device 104, and the corresponding sub-modules ofthese functions may be located within client device 104 rather thanserver system 108. In some embodiments, at least some of the functionsof client device 104 are performed by server system 108, and thecorresponding sub-modules of these functions may be located withinserver system 108 rather than client device 104. Client device 104 andserver system 108 shown in the Figures are merely illustrative, anddifferent configurations of the modules for implementing the functionsdescribed herein are possible in various embodiments.

While particular embodiments are described above, it will be understoodit is not intended to limit the application to these particularembodiments. On the contrary, the application includes alternatives,modifications and equivalents that are within the spirit and scope ofthe appended claims. Numerous specific details are set forth in order toprovide a thorough understanding of the subject matter presented herein.But it will be apparent to one of ordinary skill in the art that thesubject matter may be practiced without these specific details. In otherinstances, well-known methods, procedures, components, and circuits havenot been described in detail so as not to unnecessarily obscure aspectsof the embodiments.

Each of the above-identified elements may be stored in one or more ofthe previously mentioned memory devices, and corresponds to a set ofinstructions for performing a function described above. The aboveidentified modules or programs (i.e., sets of instructions) need not beimplemented as separate software programs, procedures, modules or datastructures, and thus various subsets of these modules may be combined orotherwise re-arranged in various implementations. In someimplementations, memory 806, optionally, stores a subset of the modulesand data structures identified above. Furthermore, memory 806,optionally, stores additional modules and data structures not describedabove.

What is claimed is:
 1. A method, comprising: at an electronic devicehaving one or more processors, and memory: deploying a frontendinformation exchange system, wherein the frontend information exchangesystem provides an input user interface configured to detect one or morein-person close-proximity interactions between a user and the frontendinformation exchange system; registering a current deployment locationof the frontend information exchange system; detecting a first in-personclose-proximity interaction between a first user and the frontendinformation exchange system at the current deployment location of thefrontend information exchange system, wherein the first in-personclose-proximity interaction specifying a first product category forwhich the first user is seeking product recommendations; in response tothe first in-person close-proximity interaction between the first userand the frontend information exchange system: automatically generating afirst product recommendation, wherein the first product recommendationincludes a first product of the first product category; detecting arespective prior purchase record of the first user for a second productin a second product category that is associated with the first productcategory; in accordance with a determination that the respective priorpurchase record for the second product indicates a purchase or deliverylocation outside of a predefined geographic region of the registeredcurrent deployment location of the frontend information exchange system,automatically augmenting the first product recommendation to include athird product from the second product category that is selected based oncharacteristics of the second product; and in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of thefrontend information exchange system, automatically refining the firstproduct recommendation to further define one or more characteristics ofthe first product based on the characteristics of the second product;and providing, through the frontend information exchange system, thefirst product recommendation to the first user after the automaticaugmenting or refining of the first product recommendation.
 2. Themethod of claim 1, further including: in response to the first in-personclose-proximity interaction between the first user and the frontendinformation exchange system: detecting a respective prior purchaserecord of the first user for the first product in addition to therespective prior purchase record for the second product; and inaccordance with a determination that the respective prior purchaserecord for the first product indicates that same purchase or deliverylocation as the respective prior purchase record for the second product,automatically augmenting the first product recommendation to include thesecond product from the second product category, irrespective of whetherthe purchase or delivery location indicated in the respective priorpurchase records for the first and second products is within thepredefined geographic region of the registered current deploymentlocation of the information exchange system.
 3. The method of claim 1,wherein the first product recommendation is provided to the first userafter the automatic augmenting of the first product recommendation toinclude the third product from the second product category that isselected based on the characteristics of the second product, and whereinthe method further includes: detecting, through the frontend informationexchange system, a user input rejecting the third product from thesecond product category that is included in the first productrecommendation; and in response to detecting the user input rejectingthe third product from the second product category that is included inthe first product recommendation, revising the first productrecommendation to remove the third product from the second productcategory and include a fourth product from a respective product categoryfor which the first user has no prior purchase record.
 4. The method ofclaim 1, wherein the first product recommendation is provided to thefirst user after the automatic refining of the first productrecommendation to further define one or more characteristics of thefirst product based on the characteristics of the second product, andwherein the automatic refining includes: determining a size category ofthe first product; determining a size category of the second product; inaccordance with a determination that the size category of the firstproduct is distinct from the size category of the second product,defining the one or more characteristics of the first product based onthe characteristics of the second product in accordance with apre-stored contrasting matching scheme; and in accordance with adetermination that the size category of the first product is the same asthe size category of the second product, defining the one or morecharacteristics of the first product based on the characteristics of thesecond product in accordance with a pre-stored harmonizing matchingscheme.
 5. The method of claim 1, including: establishing acorrespondence database that includes, for each respective productcategory of a plurality of product categories, a corresponding set offortune forecasting parameters that are determined to be compatible withthe respective product category; in response to the first in-personclose-proximity interaction between the first user and the frontendinformation exchange system: identifying a set of fortune forecastingparameters that are currently valid for the first user; and inaccordance with the correspondence database, identifying a third productcategory from the plurality of product categories that corresponds tothe set of fortune forecasting parameters that are currently valid forthe first user; and augmenting the first product recommendation toinclude one or more products from the third product category with thefirst product.
 6. The method of claim 1, wherein the first in-personclose-proximity interaction specifying a first product quantity for thefirst product category that the first user intends to purchase, andwherein the method includes: automatically generating a digital dealthat bundles the first product of the first product quantity and thethird product of a second product quantity that is selected based on thefirst product quantity.
 7. The method of claim 1, wherein selecting thesecond product quantity includes: in accordance with the determinationthat the respective prior purchase record for the second productindicates a purchase or delivery location outside of the predefinedgeographic region of the registered current deployment location of theinformation exchange system, selecting the second product quantity basedon the first product quantity without regard to a prior purchasequantity of the second product indicated in the respective priorpurchase record; and in accordance with a determination that therespective prior purchase record for the second product indicates apurchase or delivery location within the predefined geographic region ofthe registered current deployment location of the information exchangesystem, selecting the second product quantity based on the first productquantity and the prior purchase quantity of the second product indicatedin the respective prior purchase record.
 8. A system, comprising: one ormore processors; and memory storing instructions which when executed bythe one or more processors, cause the processors to perform operationscomprising: deploying a frontend information exchange system, whereinthe frontend information exchange system provides an input userinterface configured to detect one or more in-person close-proximityinteractions between a user and the frontend information exchangesystem; registering a current deployment location of the frontendinformation exchange system; detecting a first in-person close-proximityinteraction between a first user and the frontend information exchangesystem at the current deployment location of the frontend informationexchange system, wherein the first in-person close-proximity interactionspecifying a first product category for which the first user is seekingproduct recommendations; in response to the first in-personclose-proximity interaction between the first user and the frontendinformation exchange system: automatically generating a first productrecommendation, wherein the first product recommendation includes afirst product of the first product category; detecting a respectiveprior purchase record of the first user for a second product in a secondproduct category that is associated with the first product category; inaccordance with a determination that the respective prior purchaserecord for the second product indicates a purchase or delivery locationoutside of a predefined geographic region of the registered currentdeployment location of the frontend information exchange system,automatically augmenting the first product recommendation to include athird product from the second product category that is selected based oncharacteristics of the second product; and in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of thefrontend information exchange system, automatically refining the firstproduct recommendation to further define one or more characteristics ofthe first product based on the characteristics of the second product;and providing, through the frontend information exchange system, thefirst product recommendation to the first user after the automaticaugmenting or refining of the first product recommendation.
 9. Thesystem of claim 8, wherein the operations further include: in responseto the first in-person close-proximity interaction between the firstuser and the frontend information exchange system: detecting arespective prior purchase record of the first user for the first productin addition to the respective prior purchase record for the secondproduct; and in accordance with a determination that the respectiveprior purchase record for the first product indicates that same purchaseor delivery location as the respective prior purchase record for thesecond product, automatically augmenting the first productrecommendation to include the second product from the second productcategory, irrespective of whether the purchase or delivery locationindicated in the respective prior purchase records for the first andsecond products is within the predefined geographic region of theregistered current deployment location of the information exchangesystem.
 10. The system of claim 8, wherein the first productrecommendation is provided to the first user after the automaticaugmenting of the first product recommendation to include the thirdproduct from the second product category that is selected based on thecharacteristics of the second product, and wherein the operationsfurther include: detecting, through the frontend information exchangesystem, a user input rejecting the third product from the second productcategory that is included in the first product recommendation; and inresponse to detecting the user input rejecting the third product fromthe second product category that is included in the first productrecommendation, revising the first product recommendation to remove thethird product from the second product category and include a fourthproduct from a respective product category for which the first user hasno prior purchase record.
 11. The system of claim 8, wherein the firstproduct recommendation is provided to the first user after the automaticrefining of the first product recommendation to further define one ormore characteristics of the first product based on the characteristicsof the second product, and wherein the automatic refining includes:determining a size category of the first product; determining a sizecategory of the second product; in accordance with a determination thatthe size category of the first product is distinct from the sizecategory of the second product, defining the one or more characteristicsof the first product based on the characteristics of the second productin accordance with a pre-stored contrasting matching scheme; and inaccordance with a determination that the size category of the firstproduct is the same as the size category of the second product, definingthe one or more characteristics of the first product based on thecharacteristics of the second product in accordance with a pre-storedharmonizing matching scheme.
 12. The system of claim 8, wherein theoperations include: establishing a correspondence database thatincludes, for each respective product category of a plurality of productcategories, a corresponding set of fortune forecasting parameters thatare determined to be compatible with the respective product category; inresponse to the first in-person close-proximity interaction between thefirst user and the frontend information exchange system: identifying aset of fortune forecasting parameters that are currently valid for thefirst user; and in accordance with the correspondence database,identifying a third product category from the plurality of productcategories that corresponds to the set of fortune forecasting parametersthat are currently valid for the first user; and augmenting the firstproduct recommendation to include one or more products from the thirdproduct category with the first product.
 13. The system of claim 8,wherein the first in-person close-proximity interaction specifying afirst product quantity for the first product category that the firstuser intends to purchase, and wherein the method includes: automaticallygenerating a digital deal that bundles the first product of the firstproduct quantity and the third product of a second product quantity thatis selected based on the first product quantity.
 14. The system of claim8, wherein selecting the second product quantity includes: in accordancewith the determination that the respective prior purchase record for thesecond product indicates a purchase or delivery location outside of thepredefined geographic region of the registered current deploymentlocation of the information exchange system, selecting the secondproduct quantity based on the first product quantity without regard to aprior purchase quantity of the second product indicated in therespective prior purchase record; and in accordance with a determinationthat the respective prior purchase record for the second productindicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of theinformation exchange system, selecting the second product quantity basedon the first product quantity and the prior purchase quantity of thesecond product indicated in the respective prior purchase record.
 15. Anon-transitory computer-readable storage medium storing instructions,the instructions, when executed by the one or more processors, cause oneor more processors to perform operations comprising: deploying afrontend information exchange system, wherein the frontend informationexchange system provides an input user interface configured to detectone or more in-person close-proximity interactions between a user andthe frontend information exchange system; registering a currentdeployment location of the frontend information exchange system;detecting a first in-person close-proximity interaction between a firstuser and the frontend information exchange system at the currentdeployment location of the frontend information exchange system, whereinthe first in-person close-proximity interaction specifying a firstproduct category for which the first user is seeking productrecommendations; in response to the first in-person close-proximityinteraction between the first user and the frontend information exchangesystem: automatically generating a first product recommendation, whereinthe first product recommendation includes a first product of the firstproduct category; detecting a respective prior purchase record of thefirst user for a second product in a second product category that isassociated with the first product category; in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location outside of apredefined geographic region of the registered current deploymentlocation of the frontend information exchange system, automaticallyaugmenting the first product recommendation to include a third productfrom the second product category that is selected based oncharacteristics of the second product; and in accordance with adetermination that the respective prior purchase record for the secondproduct indicates a purchase or delivery location within the predefinedgeographic region of the registered current deployment location of thefrontend information exchange system, automatically refining the firstproduct recommendation to further define one or more characteristics ofthe first product based on the characteristics of the second product;and providing, through the frontend information exchange system, thefirst product recommendation to the first user after the automaticaugmenting or refining of the first product recommendation.
 16. Thecomputer-readable storage medium of claim 15, wherein the operationsfurther include: in response to the first in-person close-proximityinteraction between the first user and the frontend information exchangesystem: detecting a respective prior purchase record of the first userfor the first product in addition to the respective prior purchaserecord for the second product; and in accordance with a determinationthat the respective prior purchase record for the first productindicates that same purchase or delivery location as the respectiveprior purchase record for the second product, automatically augmentingthe first product recommendation to include the second product from thesecond product category, irrespective of whether the purchase ordelivery location indicated in the respective prior purchase records forthe first and second products is within the predefined geographic regionof the registered current deployment location of the informationexchange system.
 17. The computer-readable storage medium of claim 15,wherein the first product recommendation is provided to the first userafter the automatic augmenting of the first product recommendation toinclude the third product from the second product category that isselected based on the characteristics of the second product, and whereinthe operations further include: detecting, through the frontendinformation exchange system, a user input rejecting the third productfrom the second product category that is included in the first productrecommendation; and in response to detecting the user input rejectingthe third product from the second product category that is included inthe first product recommendation, revising the first productrecommendation to remove the third product from the second productcategory and include a fourth product from a respective product categoryfor which the first user has no prior purchase record.
 18. Thecomputer-readable storage medium of claim 15, wherein the first productrecommendation is provided to the first user after the automaticrefining of the first product recommendation to further define one ormore characteristics of the first product based on the characteristicsof the second product, and wherein the automatic refining includes:determining a size category of the first product; determining a sizecategory of the second product; in accordance with a determination thatthe size category of the first product is distinct from the sizecategory of the second product, defining the one or more characteristicsof the first product based on the characteristics of the second productin accordance with a pre-stored contrasting matching scheme; and inaccordance with a determination that the size category of the firstproduct is the same as the size category of the second product, definingthe one or more characteristics of the first product based on thecharacteristics of the second product in accordance with a pre-storedharmonizing matching scheme.
 19. The computer-readable storage medium ofclaim 15, wherein the operations include: establishing a correspondencedatabase that includes, for each respective product category of aplurality of product categories, a corresponding set of fortuneforecasting parameters that are determined to be compatible with therespective product category; in response to the first in-personclose-proximity interaction between the first user and the frontendinformation exchange system: identifying a set of fortune forecastingparameters that are currently valid for the first user; and inaccordance with the correspondence database, identifying a third productcategory from the plurality of product categories that corresponds tothe set of fortune forecasting parameters that are currently valid forthe first user; and augmenting the first product recommendation toinclude one or more products from the third product category with thefirst product.
 20. The computer-readable storage medium of claim 15,wherein the first in-person close-proximity interaction specifying afirst product quantity for the first product category that the firstuser intends to purchase, and wherein the method includes: automaticallygenerating a digital deal that bundles the first product of the firstproduct quantity and the third product of a second product quantity thatis selected based on the first product quantity.